Before a software application can be deployed, it is typically tested to be sure it behaves as the developer intended. Where the application requires user input, such as through a graphical user interface (GUI), various input possibilities must be tested to ensure the application responds appropriately to the information provided. This also must be done if a GUI is changed, or added to the application by developers or by users that may need to tailor the application to suit their particular needs. One method of testing involves manual interaction with the GUI and/or supplying inputs to GUI objects to test their response. However, manual testing can be highly inefficient, especially where a large number of input possibilities are involved. Thus, in a field known as testing automation, software testers develop test scripts to automatically cycle through and provide numerous inputs in order to analyze the functionality of an application. Unfortunately, these test scripts can become as complex as the application source code they are testing and to reference GUI objects within the application requires knowledge of, and access to the source code underlying the application.
The challenge is communicating to the application what it is to do in a non-technical manner. Manually, a user can do this by interacting with the GUI objects, such as by entering data in a textbox or clicking on a command button. However, to automate this process so that it can cycle through a series of inputs without manual entry, test scripts have traditionally been required to identify the specific GUI object that is to be selected or that is to receive input by using the name given that GUI object within the application source code. For example, if a test script is to cycle through a series of numbers from 1 to 100 entered into a text box, the script must reference the text box with its name in the source code, which may be assigned according to a programming lexicon known only to the original programmer. If the source code is not available, as is typically the case, testing automation becomes complicated and requires a technically savvy tester.
The present invention is provided to create annotations for each unannotated GUI object based on the properties and positions of other GUI objects in a GUI. The GUI objects can include the unannotated GUI objects, candidate GUI objects, redundant GUI objects, and self-described GUI objects. The candidate GUI objects will contain text strings which can be used as the basis for creating annotations for the unannotated GUI objects. Nearby candidate GUI objects that are in the vicinity of an unannotated GUI object can be determined by the present invention, based on the locations and bounding rectangles of the unannotated GUI object and the candidate GUI objects, and the role and a precedence criteria of the unannotated GUI object. A best candidate GUI object can be selected from the nearby candidate GUI objects based on the precedence criteria. The text string included in the best candidate GUI object can then be used as the annotation for the unannotated GUI object.
By creating annotations for the unannotated GUI objects, the present invention can be used in conjunction with automated testing and documentation of a GUI, particularly by non-technical users. For instance, the present invention can help non-technical users test and document browser and legacy applications written in a variety of programming languages without reference to or needing access to the underlying source code, such as through the use of software including TestDrive from Original Software, for example. The present invention can simplify the commands used by non-technical users in creating a series of test automation steps in a test script, or in generating documentation of a GUI. Other features and advantages are provided by the following description and drawings.